Batch processing is one type of data processing method of a computer. In batch processing, data is gathered for a certain period of time or by a certain amount, and is processed collectively as one task. As a large amount of data is processed collectively, when the processing is terminated abnormally in the middle of the processing, the cost tends to be higher if the processing is re-executed from the beginning. As such, a system has been proposed and put into practice use, in which the execution status of a task is stored in a storage device from time to time, and if the processing is terminated abnormally, the processing is re-executed from the middle of the task (for example, see JP 9-282192 A (Patent Document 1)).
In batch processing, simple processing is often executed repeatedly. In enterprise batch processing, for example, the same processing is performed on respective input item (data) strings, and the results are output. As such, there is a form in which task processing is performed in a distributed manner using a plurality of nodes (computers) (for example, see JP 10-326201 A (Patent Document 2)). Such a processing form is called distributed task processing, or simply called distributed processing, and a system executing it is called a distributed processing system. In a distributed processing system, execution statuses of a task in the respective nodes are often collected, stored, and managed in a persistent storage device in preparation for recovery processing or the like.
Patent Document 1: JP 9-282192 A
Patent Document 2: JP 10-326201 A
Here, consideration is given to a distributed processing system which is configured of a management node and a plurality of execution nodes as shown in FIG. 12, in which execution statuses of a task are collectively managed in a persistent storage device (execution status persistent storage device) of the management node. In this distributed management system, the management node divides an input task and allocates to the respective execution nodes. Each of the execution nodes executes the allocated task, and upon completion of the execution, transmits an execution status update request to the management node. In accordance with the received execution status update request, the management node updates the execution status information of the task stored in the persistent storage device. Thereby, the execution statuses of the task executed by a plurality of execution nodes are stored in the persistent storage device, and can be stored persistently as consistent information.
In the distributed management system as described above, if the respective execution nodes have the same processing performance, when the management node divides the input task equally and allocates to the respective execution nodes as shown in FIG. 13, the respective execution nodes complete execution of the task at almost the same time. It should be noted that in FIG. 13, small rectangle pieces show items to be processed. Accordingly, as shown in FIG. 14, there is a case where all of the execution nodes transmit execution status update requests to the management node at almost the same time, whereby a large number of execution status update requests arrive at the management node at once. As such, in the management node having a limitation in the number of execution status update requests which can be processed simultaneously, it is impossible to receive and process all of the execution status update requests. Thus, it is difficult to perform coherent execution status update processing, which brings difficulty in storing task execution statuses in a concentrated and consistent manner.